Method and System for Image Processing to Classify an Object in an Image

ABSTRACT

In an image processing method, an object is located within an image. An area around the object is determined and divided into at least first and second portions based upon image information within the area. The object can then be classified based upon both image information in the first portion of the area and image information in the second portion of the area.

This is a continuation application of U.S. patent application Ser. No.12/686,902, entitled “Player Team Classification-Based InteractiveService for Soccer Game Programs in an IPTV System” which was filed onJan. 13, 2010, which application claims the benefit of U.S. ProvisionalApplication No. 61/144,380, filed on Jan. 13, 2009, entitled “PlayerTeam Classification-Based Interactive Service for Sports Game Programsin an IPTV System” and also claims the benefit of U.S. ProvisionalApplication No. 61/144,386, filed on Jan. 13, 2009, entitled “ASemi-Supervised Method For Learning and On-Line Updating Playfield Modelin Sports Videos.” All three of these applications are incorporatedherein by reference in their entireties.

TECHNICAL FIELD

The present invention relates generally to image processing, and inparticular embodiments, to a method and system for image processing toclassify an object in an image.

BACKGROUND

Systems and methods have been developed for defining an object in videoand for tracking that object through the frames of the video. In variousapplications, a person may be the “object” to be tracked. For example,sports images are interested in following the actions of a person suchas the players and/or the referees.

Players and referees are displayed in sports videos. Localization andlabeling of them can be done in IPTV systems so that a regular TVbroadcast (MPEG-2/-4) is augmented with additional information (MPEG-7encoded) that defines those objects in the video, along with additionalcontent to be displayed when they are selected. Specification of objectswith additional content (metadata) is usually implemented by anauthoring tool that includes such functions as extraction of shots andkey frames, specification of the interactive regions, and tracking ofthe specified regions to get the region locations in all frames.

Team classification-based interactive services by clicking the player inhypervideo or iTV has been discussed. Team information search andretrieval and team data (statistics results, articles and other media)can be linked assuming the player can be localized by the interactionservice system. Various methods for locating the players/referees can besplit in two groups. The first group makes use of fixed cameras (usuallythey are calibrated in advance) in a controlled environment while thesecond group uses only regular broadcasting videos. While the former canprovide better performance, the latter are more flexible. In the secondgroup, some approaches tried to overcome difficulties by finding theplayfield first, using color segmentation and post-processing withmorphological operations, such as connected component analysis, in orderto limit the search area.

SUMMARY OF THE INVENTION

In accordance with a first embodiment of the present invention, an imageprocessing method is performed, e.g., on a processor. An object islocated within an image, such as a video or still image. An area aroundthe object is determined and divided into at least first and secondportions based upon image information within the area. The object canthen be classified based upon both image information in the firstportion of the area and image information in the second portion of thearea.

In another embodiment, an interactive television system includes anauthoring tool configured to receive a video image, locate an objectwithin the image, divide an area around the object into first and secondportions; and generate metadata based upon first image informationwithin the first portion and upon second image information within thesecond portion. An aggregator is configured to receive the video imageand metadata and generate a video stream that is enhanced with themetadata and a delivery system is configured to transmit the videostream that is enhanced with the metadata.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention, and theadvantages thereof, reference is now made to the following descriptionstaken in conjunction with the accompanying drawings, in which:

FIG. 1 illustrates a block diagram of a team classification-basedinteractive system, according to one embodiment of the presentinvention;

FIG. 2 illustrates a flow diagram for playfield model-basedplayer/referee localization, according to one embodiment of the presentinvention;

FIG. 3 illustrates an example image of a vertical cutting line in aplayer blob, according to one embodiment of the present invention;

FIG. 4 illustrates block diagram of a interactive television system,according to one embodiment of the present invention; and

FIGS. 5-7 illustrate screen shots of an example of an interactivetelevision system.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The making and using of the presently preferred embodiments arediscussed in detail below. It should be appreciated, however, that thepresent invention provides many applicable inventive concepts that canbe embodied in a wide variety of specific contexts. The specificembodiments discussed are merely illustrative of specific ways to makeand use the invention, and do not limit the scope of the invention.

Team classification of player/referee is used to distinguish theiridentities as Team A or Team B or Referee. Issues for this task includethe selection of features and the clustering method for matching.Template and histogram methods have been used. Feature selection isbased on discrimination of different classes, robustness andcomputational cost. Histograms are a good trade-off between theserequisites. Clustering methods can be supervised or unsupervised. Thepresent invention improves the efficiency of feature extraction anddiscrimination in histogram matching simultaneously.

In a first embodiment, the present invention discloses a sports teambased interactive service for IPTV systems, including real time and ondemand video delivery. For example, a sporting event video is processedand the team/referee visual objects are analyzed and categorized in realtime. A multi-histogram matching scheme and a method for separating theplayer/referee blob (obtained by playfield model-based segmentation)into two parts (upper and lower) via a discriminate function areproposed. The proposed scheme achieves good classification accuracy withlow computational complexity. The team classification-based interactionfunctions include team information search and retrieval and team data(statistics result, articles and other media) linking when the player isclicked. The proposed team classification method still has a promisingpotential usage in sporting events and tactics analysis as well as aninteractive service for sports programs in IPTV systems.

In other embodiments, a proposed team classification-based interactiveservice for sports programs in an IPTV system is provided. In anotherembodiment, a method for team classification of player/referee in soccergame videos based on a multi-histogram matching scheme, which offersbetter classification accuracy with low computational complexity, isprovided. In yet another embodiment, a method can be used to separatethe player/referee blob (obtained by playfield model-based segmentation)into two parts (upper and lower) by a proposed discriminate function.

In hyperlinked video, objects are selectable resulting in an associatedaction, akin to linked rich media content about the objects of interest.Possible venues for hyperlinked video include broadcast TV, streamingvideo and published media such as DVD. Hyperlinked video offers newinteraction possibilities with streaming media.

Interactive TV (iTV) is a popular application area of hyperlinked videowith the convergence between broadcast and network communications. Forexample, the European GMF4iTV (Generic Media Framework for InteractiveTelevision) project has developed such a system where active videoobjects are associated to metadata information, embedded in the programstream at production time and can be selected by the user at run time totrigger the presentation of their associated metadata. Another EuropeanPorTiVity (Portable Interactivity) project is developing andexperimenting with a complete end-to-end platform providing Rich MediaInteractive TV services for portable and mobile devices, realizingdirect interactivity with moving objects on handheld receivers connectedto DVB-H (broadcast channel) and UMTS (unicast channel).

IPTV (Internet Protocol Television) is a system where a digitaltelevision service is delivered using Internet Protocol over a networkinfrastructure, which may include delivery by a broadband connection. AnIP-based platform also allows significant opportunities to make the TVviewing experience more interactive and personalized. Interactive TVservices will be a key differentiator for the multitude of IPTVofferings that are emerging. Interactivity via a fast two-way connectionwill lift IPTV ahead of today's television.

Aspects of the present disclosure relate to a scenario related to a richmedia interactive TV application, for example, an IPTV application. Thefocus is interaction with moving objects in sports programs. Based ondirect interaction with certain objects, the TV viewer can retrieve andlink rich media content about objects of interest. The term “television”or “TV” is used to denote any video image displayed to a user. Forexample, this image can be displayed on a computer screen, mobile deviceor an actual television and still be within the scope of the termtelevision.

Players and referees are two examples of moving objects in sportsvideos. Localization and labeling of them is useful for interactiveservices in IPTV systems, so that a regular TV broadcast (MPEG-2/-4) isaugmented with additional information (MPEG-7 encoded) which definesthose objects in the video, along with additional content to bedisplayed when they are selected. Specification of objects withadditional content (metadata) can be implemented by an authoring toolthat includes such functions as extraction of shots and key frames,specification of the interactive regions, and tracking of the specifiedregions to get the region locations in all frames.

In embodiments of the present invention, a player teamclassification-based interactive service for soccer game programs in anIPTV system is proposed. FIG. 1 illustrates an overview of this IPTVinteraction service system 100.

The interaction is based on the combination of information prepared onthe IPTV server side and real time team classification on the IPTVclient side and/or in a network middle box (such as the Content andMetadata Sources block 105 shown in FIG. 1). The information on theserver side stored as metadata in the MPEG-7 format or any otherstandard compliant (or proprietary) format. The information describesthe playing field, both team multi-histogram models and related mediainformation about the teams. A functional unit within a network middlebox or the client side does the real-time team classification based onon-line player/referee feature extraction and the MPEG-7 metadata,presenting the related media information on TV screen for userinteraction.

The system can be applied to a number of sports. For example, sportssuch as soccer, football, basketball, baseball, hockey, cricket andothers can utilize the concepts described herein.

FIG. 1 discloses a team classification-based IPTV interaction system100, in accordance with aspects of the present invention. It is notedthat the above specific configuration of FIG. 1 is only one of the manypossible configurations. For instance, the playing field information aswell as the team player color statistics can be implemented at a networknode or the client side as well.

In the embodiment, the user is registered with the IMS infrastructure.The TV content is to be enhanced with metadata information for theplayfield description and team target models represented as themulti-color histograms. The IPTV client is enhanced with such a service,which implies an environment to run additional services and respectivelyexecute advanced program code on the IPTV client for on-line playerlocalization (segmentation or tracking).

FIG. 1 further illustrates a sample system architecture. Under thisarchitecture, a basic action flow may include: the user 160, whoregisters to request an interactive service and makes use of theservice. The user 160 is able to click on a player/referee to localizethe object of interests (segmentation/tracking) and get the teaminformation and team related metadata on remote control. The IMS-basedIPTV Client 155 (such as Set-Top-Box or PDA) is responsible forproviding the user 160 with the necessary functionality to take use ofthe interaction, e.g., real time player/referee localization and viewingthe additional content.

The IPTV Service Control Function 150 manages all user-to-content andcontent-to-user relationships and controls the Content Delivery andStorage 140 and the Content Aggregator 110. The IPTV ApplicationFunction 145 supports various service functions and provides aninterface to the user 160 to notice the IPTV service information andaccept the service request of the user (such as registration orauthentication). The IPTV Application Function 145, in conjunction withthe Service Control Function 150 provides users with the value addedservices they request.

The Content Preparation 130 sends a content distribution request to theContent Delivery Control 135. The Content Delivery Control 135 producesa distribution task between Content Preparation 130 and the ContentDelivery and Storage 140 according to the defined distribution policywhen it receives the request of content distribution. The ContentDelivery and Storage 140 delivers aggregated and metadata-enhancedcontent to the user 160, and may perform player localization and teamclassification in implementations where these task are not performed atthe IPTV Client 155.

The Content Aggregator 110 links the content 120 to the metadata 125 viathe Authoring Tool 115 and aggregates content that is enhanced withmetadata information for interactive service purposes. The AuthoringTool 115 runs play field learning and team model acquisition andgenerates the MPEG-7 metadata.

Although the present invention targets interactive services in IPTVsystems, the invention is not so limited. The proposed scheme can beused in other video delivery systems with improved accuracy and lowcomputational complexity.

The player/referee localization can be performed in a number of ways.For example, various methods for locating the players/referees can besplit in two groups. The first group makes use of fixed cameras (usuallycalibrated in advance) in a controlled environment. Such a technique istaught by M. Xu, J. Orwell, G. Jones, “Tracking football players withmultiple cameras”. ICIP 2004, pp. 2909-2912, the content of which isincorporated herein by reference. The second group uses only regularbroadcasting videos. While the former can provide better performance,the latter are more flexible. In the second group, difficulties inlocalization can be overcome by finding the playfield first, using colorsegmentation and post-processing with morphological operations, such asconnected component analysis, in order to limit the search area.

FIG. 2 shows a typical framework for playfield model-basedplayer/referee localization. This figure illustrates playfieldmodel-based player/referee localization, according to embodiments of thepresent invention. Team classification of player/referee attempts todistinguish each person as Team A or Team B or Referee. While thediscussion is related to a player/referee, it should be understood thatthe invention can be utilized to recognize other objects as well. Theseobjects can be people, such as player, referee, coach, commentator,mascot, fan or other. Alternatively, the object can be a non-humananimal, such as a horse in a horse race or a mascot animal at a collegefootball game, or an inanimate object such as a ball, a field indicatoror any of the other innumerable objects at a sporting event.

The issues for the localization task are selection of features and theclustering method for matching. In one aspect, the present inventionimproves the efficiency of feature extraction and discrimination inhistogram matching simultaneously. Embodiments adopt a multi (e.g.,two)-histogram based matching method to classify the players andreferees in game videos.

FIG. 2 shows a typical framework for playfield model-basedplayer/referee localization. As illustrated in diagram 200, theframework can be divided into two portions, playfield extraction 205 andobject detection 230. The next step, discussed with respect to thediagram of FIG. 3, is object classification.

The playfield extraction 205 includes playfield pixel detection 210,connected component analysis 215, morphological filtering (e.g.,dilation, erosion) 220 and size filtering 225. Further details onplayfield extraction will now be discussed.

The playfield can be used for analyzing several types of sports videos,such as soccer, football, baseball and tennis. For example, theplayfield is grass for a soccer, baseball or football game. While thecolor of grass is generally green, this color can vary depending on theindividual playfield, the presence of shadows or the viewing angle. Inanother example, the playfield is ice for a hockey game, but similarissues will occur.

Embodiments of the invention will now be described with to respectsoccer. It is understood that the same concepts apply to other sports aswell.

Due to the nature of the soccer game, there are many wide shots wherethe majority of the image is the playfield. Based on this observation,an unsupervised segmentation technique can obtain the playfield model.However, the playfield area in any frame is not always big enough tomake the assumption of dominant color valid. Therefore, supervisedmethods for learning the playfield model can be used. A drawback of thesupervised methods is the requirement of many labeled data, wherehand-labeling is tedious and expensive.

In one embodiment, two options are defined. The first option is a smallset of labeled data, i.e., the pixels in a given playfield area, is usedto generate a rough playfield model with a single Gaussian or a mixtureof Gaussian distributions (for the latter one, more labeled data iscompulsory). Then, this model can be modified by collecting moreplayfield pixels based on an unsupervised method using dominant colordetection.

In a second option, one frame, where the dominant color assumption issatisfied, is selected. Then its dominant mode is extracted to generatethe initial playfield model. Like the first option, this model can bemodified by collecting more playfield pixels based on dominant colordetection.

The determination of the playfield model is discussed in greater detailin Provisional Patent Application Ser. No. 61/144,386, which isincorporated herein by reference. Further information can be derivedfrom that application.

Players and referees are foreground objects in the soccer playfield.Since soccer is a spectator sport, the play fields, the lines, the balland the uniforms of the players and referees are designed to be visuallydistinctive in color. Therefore, the framework in FIG. 2 can be appliedfor extraction or detection of player/referee blobs from the playfield.

The object detection 230 includes interior filtering 235. A comparisonis made between the pre-filtered and the filtered image data asillustrated by the XOR gate. The result can be thought of as the imagewithout the background. This result can then go through connectedcomponent analysis 240 and shape filtering 245. The shape filtering candeal with size, roundness and/or eccentricity, as examples.

Given the segmented blobs for players and referees, each will be labeledwith an identity of Team A, Team B or Referee. Sometimes two teamgoalies are classified as well. In order to do that, each team player orreferee's appearance model is used to acquire by learning the labeleddata.

Since the player's jersey is mostly discriminate from the shorts and theformer occupies more area in the player blob, two color histograms canbe used to represent the player's appearance model, with one being givena bigger weight in histogram matching. In this context, the terms jerseyand shorts are used to denote the upper portion of the player and thelower portion of the player regardless of whether the player is actuallywearing a jersey or shorts. For example, the term “shorts” includesshorts worn by a basketball and also pants worn by a baseball player.Similarly, the term “jersey” can be applied to both teams in a “shirtsv. skins” game.

FIG. 3 illustrates a player blob 300 that can be used in thediscrimination process. In this example, a soccer player 305 wears ajersey 315 and shorts 320. A rectangle 325, approximately centered atthe boundary between the jersey 315 and shorts 320, is used todistinguish the player. A vertical cutting line 310 is adjusted to bealigned between the jersey/shorts boundary.

The separation of the jersey 315 and shorts 320 in each player/refereeblob 300 into upper and lower parts (here it is assumed the playerstands approximately upright) is based on a discriminate function. Givena rectangle 325 of size w×h (width w and height h), the cutting line 310is searched around the middle position which maximizes the objectivefunction

ρ[p,q]=1.0−Σ_(M=1) ^(m)√{square root over (p _(u) q _(u))},  (1)

where the p_(u) is the color histogram of the upper part and the q_(u)is the color histogram of the lower part. m is the number of bins in thehistogram. In one embodiment, a histogram with 8×8×8 bins of color RGBcolor space can be used. The measure in equation (1) is called theBhattacharyya distance.

To accelerate the above process, the color histogram for each row in thesearch gap of the target window (rectangle 325) is calculated and soonly the row histogram is added or subtracted from histograms of theupper or lower part respectively when the cutting line 310 scrolls upand down one row per iteration. Since the position range of the cuttingline 310 is narrow, an exhaustive search is feasible to find the bestcutting line that discriminates most of the upper part from the lowerpart. For example, the search gap my include a range of less than 25% ofthe rows, or preferably, less than 10% of the rows.

Eventually two color histograms are utilized for player/refereelabeling. In appearance model learning, two histograms are generated andsaved as p_(1i) and p_(2i) for each team or referee, i.e., i=Team A,Team B and Referee. If multiple samples for each type (either teamplayer or referee), obtained from either manually segmented orplayfield-model-based segmentation, are applied, all the pixels in theupper part (jersey mainly) and the lower part (shorts mainly) of thesegmented blob are collected to build the upper and lower colorhistogram respectively. Eventually both histograms are normalized.

In testing or running the classification, for an unknown blob (extractedfrom playfield-model-based segmentation process) in the soccer gamevideos two normalized color histograms for jersey and short regions arebuilt as well, i.e., q₁ and q₂, then its label i is estimated byminimizing a weighted Bhattacharyya distance as

$\begin{matrix}{{{\min\limits_{i}{\rho \left\lbrack {\left( {p_{1\; i},p_{2\; i}} \right),\left( {q_{1},q_{2}} \right)} \right\rbrack}} = {{w \cdot \left( {1.0 - {\sum\limits_{u = 1}^{m}\; \sqrt{p_{1\; {iu}}q_{1\; u}}}} \right)} + {\left( {1 - w} \right) \cdot \left( {1.0 - {\sum\limits_{u = 1}^{m}\; \sqrt{p_{2\; {iu}}q_{2\; u}}}} \right)}}},} & (2)\end{matrix}$

where w is the weight (0<w<1.0, suggested as 0.7).

The discussion above provides details on the determination of which teama player is associated with. With additional processing, the identity ofthe player may also be determined. For example, number recognitiontechniques could be used to locate and identify the player's number. Ina typical game, the roster of the players is known so that a comparisoncould be made between the known player information and the derivedplayer information.

A specific example of an interactive television system will now bedescribed with respect to FIGS. 4-7. This example provides only one ofthe many ways that the concepts described herein could be implemented.

This scenario describes a rich media interactive television application.It focuses on new concepts for interaction with moving objects in thesport programs. Based on direct interaction with certain objects, theviewer can retrieve rich media content about objects of his choice.

The interaction is based on the combination of information prepared onthe IPTV server side and real time team classification on the IPTVclient side. The information on the server side is stored as metadata inthe MPEG-7 format and describes the play field, team templates andrelated media information about the teams. The client side does the realtime object processing and, based on the MPEG-7 metadata to do the teamclassification, presents the related media information on a screen foruser interaction.

The TV content is enhanced with metadata information for the descriptionof the field and team templates represented as the color histogram. Theuser has to be registered with the IMS infrastructure. The IPTV clienthas to be enhanced with such a service, which implies an environment torun additional services and respectively execute advanced program codeon the IPTV client for content processing and object highlighting.Charging can be used for transaction and accounting.

FIG. 4 illustrates a block diagram showing a specific configuration ofone interactive television system 400. As can be seen, this systemimplements many of the concepts discussed above with respect to FIG. 1.

Referring now to FIG. 4, the service provider 410 offers an interactivechannel and metadata information. The service provider 410 hosts theneeded network entities and provides the necessary infrastructure. AnIMS charging system provides the service provider 410 with functionalityfor accounting. This feature allows the service provider 410 to fund theoffered service.

The IPTV client 420, for example a set top box (STB), is responsible toprovide the viewer 430 with the functionality to make use of theinteraction, in terms of real time object processing, to spot highlighting of objects containing additional content, to select objects andto view additional content. The IMS based IPTV client 420 is enabledwith techniques such as real time object processing for providing theinteractive service. In another example, if the video content is notenhanced with the metadata information, the IPTV client 420 can providea user interface to the user 430 for collecting the team templates.

The user 430 makes use of the service by selecting objects, andconsuming additional content. The delivery system 440, typically ownedby the service provider 410, delivers aggregated and metadata-enhancedcontent to the user 430, provides trick functions and highly efficientvideo and audio coding technologies.

The content aggregator 450 links the content 460 to the metadata 470 viathe authoring tool 480. This aggregator 450 aggregates content which isenhanced with metadata information for interactive service purposes. Thecontent aggregator 450 provides the delivery system 440 with aggregatedcontent and attaches them with enhanced content. Therefore MPEG7 asstandard for multimedia metadata descriptions should be considered.

The authoring tool 480 disposes algorithms for field learning and teamtemplate acquisition in video streams and an MPEG-7 metadata generator.

In operation of the system 400, the user 430 registers with the serviceprovider 410 and requests the desired service. For this example, theuser 430 is able to click on a player to start tracking the player andget the team information about his/her team information and relatedvideo by clicking on the appropriate colored button on remote control.

In response to the request from the user 430, the service provider 410causes the aggregator 450 to prepare the enhanced content. In doing so,the aggregator 450 communicates with the authoring tool 480, whichprocesses the content image and enhances the content 460 with themetadata 470. The aggregator 450 can then provide the aggregated contentto the delivery system 440.

The delivery system 440 forwards the enhanced content to the IPTV client420, which interacts with the user 430. The user 430 also providesstream control to the delivery system 440, either via the IPTV client420 or otherwise.

Features of each of the functional units shown in FIG. 4 will bedescribed in the following paragraphs.

Features of the service provider 410 include:

-   -   Interpret metadata provided by the content provider to inject        interactive elements    -   Connect to IMS network (e.g., over the ISC interface)    -   Have awareness of the content    -   Provide service triggering based on a Public Service Identifier        (PSI)    -   Accept and execute requests from user 430    -   Control the aggregator 450 in case of inband signaling    -   Control the delivery system 440 for forwarding the content from        the aggregator 450 to the IPTV client 420

Features of the IPTV client 420 include

-   -   IMS enabled client (STB)    -   Audio and video rendering support    -   Basic codec support such as AC3 for audio and H.264/VC1 for        video    -   RTP de-packetizing (based on RTP profiles) support for supported        audio and video codecs (e.g., H.264)    -   Real time object processing for object detection and object        tracking    -   Application logic (data engine) processing the segmentation and        editing of MPEG-7 metadata information (MPEG? Decoder)    -   Overlay-rendering support (display engine) for object        highlighting as shown in the sample and interactive menu.    -   Display engine for additional content related to the selected        object (picture in picture rendering, online shop, web portal,        reuse of MHP, others)

Features of the user 430 include:

-   -   make use of the IMS enabled client 420    -   request the content    -   needs trick modes for stream control    -   select the video object via remote control    -   retrieve additional information

Features of the delivery system 440 include:

-   -   Provide content delivery to the IPTV client 420 via unicast or        multicast channel    -   Transcode    -   Adapt content    -   Connect to the IMS core    -   Enable the IPTV client 420 to trigger media processing and        content delivery    -   Support for trick functions; RTSP support    -   Inband (DVB-MPEG TS multiplex) and/or outband (linked content        available on media ports) transport of metadata

Features of the aggregator 450 include:

-   -   Aggregate the highlight streams enhanced with metadata    -   Interface with Authoring Tool 480 (application server)    -   Prepare aggregated content for delivery

Features of the authoring tool 480 include:

-   -   Be linked to content    -   Run field learning algorithms to learn the field.    -   Run the object detection algorithm to collect the team template        information.    -   Generate MPEG-7 metadata which holds information about the field        and team templates

FIGS. 5-7 provide examples of screen shots from the user's perspective.For example, in FIG. 5, a player is identified (in this case as beingfrom Spain). This identification can occur by the user clicking on theplayer.

In FIG. 6, a new window was opened in response to the indication of theplayer from Spain. In this example, a search was performed for websitesrelated to the Spanish team. Similarly, FIG. 7 shows an example where asearch was performed for videos related to the Spanish team. While ageneral Internet search is illustrated, it is understood thatproprietary information could be provided as well. For example, thewindow can provide a list of highlight videos, statistics or otherinformation about the team. This information could be publicly availableor available only to subscribers.

Aspects of the invention have been described in the context of specificexamples. It is understood, however, that the invention is not limitedto just these examples. For example, the invention has been discussedwith respect to video images. The concepts described herein couldequally apply to still images. The concepts have also been applied tosports images. Any other images, whether photographic, drawn, computergenerated or other, can also utilize concepts described herein.

While this invention has been described with reference to illustrativeembodiments, this description is not intended to be construed in alimiting sense. Various modifications and combinations of theillustrative embodiments, as well as other embodiments of the invention,will be apparent to persons skilled in the art upon reference to thedescription. It is therefore intended that the appended claims encompassany such modifications or embodiments.

What is claimed is:
 1. An image processing method comprising: receivingan image; locating a foreground object within a frame of the image, theforeground object being depicted by a plurality of pixels; determiningan area around the foreground object within the frame of the image;dividing, using a processor, the area into at least first and secondportions based upon image color information within the area, wherein thefirst portion comprises a first group of the pixels including a portionof the foreground object and the second portion comprises a second groupof the pixels including a different portion of the foreground object,the first group of the pixels having a first color characteristic andthe second group of the pixels having a second color characteristicdifferent from the first color characteristic; and after the dividing,classifying the foreground object within the frame of the image basedupon the first color characteristic of the first portion of the area andthe second color characteristic of the second portion of the area, theclassifying being performed by comparing the first color characteristicwith first stored color characteristic information and comparing thesecond color characteristic with second stored color characteristicinformation.
 2. The method of claim 1, wherein receiving the imagecomprises receiving a video image.
 3. The method of claim 2, whereinreceiving the image comprises receiving an internet protocol television(IPTV) image.
 4. The method of claim 1, wherein dividing the area intoat least first and second portions comprises locating a line at a colorboundary within the area.
 5. The method of claim 4, wherein the areacomprises a rectangle and the line comprises a line such that the areais divided into only first and second portions.
 6. The method of claim1, wherein the first portion is represented by a first color histogramand the second portion is represented by a second color histogram andwherein the area is divided into the first and second portions basedupon the first and second color histograms.
 7. The method of claim 6,wherein dividing the area into at least first and second portionscomprises locating a line at a color boundary utilizing a Bhattacharyyadistance.
 8. The method of claim 7, wherein locating the line comprisesutilizing a weighted Bhattacharyya distance.
 9. The method of claim 6,wherein the area comprises a plurality of rows and wherein dividing thearea into at least first and second portions comprises calculating aplurality of row color histograms for rows in the area and comparingeach row color histograms with first and/or second color histograms todetermine a location of a boundary between the first portion and thesecond portion.
 10. The method of claim 9, wherein the area comprises arectangle and wherein calculating a plurality of row color histogramscomprises calculating row color histograms for only a small number ofrows, the small number of rows comprising less than ten percent of atotal number of rows in the rectangle.
 11. The method of claim 1,wherein the foreground object comprises a sports player having a jerseyand shorts and wherein classifying the foreground object comprisesclassifying the foreground object based upon a color of the jersey and acolor of the shorts.
 12. The method of claim 1, wherein classifying theforeground object comprises determining an affiliation of the foregroundobject with an organization.
 13. A method for classifying individuals ina video, the method being performed on a processor and comprising:receiving a video image; locating a player/referee within the videoimage, the player/referee comprising a plurality of pixels in a frame ofthe video image; determining an area around the player/referee, whereinthe area distinguishes the player/referee from other players/referees inthe video image; dividing the area into an upper portion that includes afirst group of the pixels that depict an upper portion of theplayer/referee and a lower portion that includes a second group of thepixels that depict a lower portion of the player/referee; determiningcolor information for the upper portion and color information for thelower portion; performing a first comparison of the color informationfor the upper portion with known top color information; performing asecond comparison of the color information for the lower portion withknown bottom color information; and identifying a characteristic of theplayer/referee based upon the first comparison and the secondcomparison, the characteristic related to affiliation.
 14. The method ofclaim 13, wherein the video image is a sports video, wherein the knowntop color information comprises jersey color information and wherein theknown bottom color information comprises shorts color information. 15.The method of claim 14, wherein identifying the characteristic of theplayer/referee comprises determining a team of the player/referee. 16.The method of claim 13, wherein identifying the characteristic of theplayer/referee comprises determining an identity of the player/referee.17. The method of claim 13, wherein receiving the video image comprisesreceiving a broadcast video image and wherein identifying thecharacteristic of the player/referee is performed substantially inreal-time with the broadcast video image.
 18. An interactive televisionsystem comprising: an authoring tool configured to receive a broadcastvideo image, locate a plurality of pixels depicting an object within thebroadcast video image, determine an area around the object that segmentsthe object from other objects in the broadcast video image, divide thearea around the object into a first group of the pixels including aportion of the object and a second group of the pixels including adifferent portion of the object, determine first color informationrelated to the first group of the pixels and second color informationrelated to the second group of the pixels, and generate metadataincluding an affiliation of the object based upon the first colorinformation within the first group of the pixels and the second colorinformation within the second group of the pixels; an aggregatorconfigured to receive the broadcast video image and metadata andgenerate a video stream that is enhanced with the metadata; and adelivery system configured to transmit the video stream that is enhancedwith the metadata.
 19. The interactive television system of claim 18,further comprising an interactive television client coupled to receivethe video stream that is enhanced with the metadata from the deliverysystem.
 20. The interactive television system of claim 18, wherein thebroadcast video image comprises an IPTV image.
 21. An apparatuscomprising: a microprocessor; and a non-transitory computer-readablestorage medium with an executable program stored thereon, wherein theexecutable program is configured to instruct the microprocessor to:locate a foreground object within a frame of an image, the foregroundobject being depicted by a plurality of pixels; determine an area aroundthe foreground object within the frame of the image; divide the areainto at least first and second portions based upon image colorinformation within the area, wherein the first portion comprises a firstgroup of the pixels including a portion of the foreground object and thesecond portion comprises a second group of the pixels including adifferent portion of the foreground object, the first group of thepixels having a first color characteristic and the second group of thepixels having a second color characteristic different from the firstcolor characteristic; and classify the foreground object within theframe of the image based upon the first color characteristic of thefirst portion of the area and the second color characteristic of thesecond portion of the area, wherein classifying the foreground objectcomprises comparing the first color characteristic with first storedcolor characteristic information and comparing the second colorcharacteristic with second stored color characteristic information.